A database may store records representing multiple different types of objects. For example, a customer relationship management (CRM) database of a CRM system may store records representing event objects, task objects, account objects, contact objects, etc. Services may change data within the database and these changes may include modifying, creating, or deleting records. For example, when a user creates a meeting through a CRM service of the CRM system, a record corresponding to an event object is created in the corresponding CRM database. To create this record, the CRM service sends a request to the CRM database to cause the creation of the record.
Records in a database may be used by external systems for various functions. For example, an external system may generate metrics at least partially based on records within a CRM database. To facilitate the generation of metrics based on records within the CRM database, a transaction observer of the CRM system may monitor all changes to the CRM database and generate a message for each of these changes. The messages may thereafter be made available, via an enterprise messaging platform and a streamer, to the external system such that external system can update their metrics based on a current state of data in the CRM database. Since the transaction observer of the CRM system is not collocated with the external system, each message must be transported over a network. As there may be thousands or millions of changes to the CRM database each day and thus an equally large number of messages representing these changes, the bandwidth occupied by messages reflecting changes to the CRM database may be very large and burdensome. Further, the external system may update the metrics based on each individual message. After each update, the metrics may be made available and eventually transported to consumers, including the CRM system. Thus, the processing power devoted by the external system to messages reflecting changes to the CRM database and bandwidth devoted to transporting the updated metrics may be equally large and burdensome.
Although the description above was presented in relation to CRM systems, similar problems are encountered by all databases that demonstrate a current state of data to external systems using messages that reflect changes to the databases.